Impacts of Cmos Scaling on the Analog Design

The advancement of the CMOS fabrication process has pushed the CMOS transistor scaling to the sub-100nm node. While process fabrication and logic designers advocated CMOS scaling consistent with Moore's Law, circuit engineers are struggling with the high leakage current, low power supply, and high power consumption. For the analog circuit designer, things become even worse due to the loss in dynamic range.The objective of this research was to investigate the impacts of the CMOS scaling on the analog design and proposed analog scaling rule: the overdrive voltage should scale at the same rate or faster than the supply voltage to maintain a power settling product efficiency which is constant or improving. To avoid a power consumption penalty, the final specifications for the analog power supply will stall at a voltage of near 1.1V, with an overdrive voltage of 0.1V. Device thresholds must be limited to an approximate voltage 0.3V for analog designs. Due to the reducing self-gain of the transistor from the scaling, multistage OTA topologies should be adopted to achieve high gain and high bandwidth. Different OTA topologies were analyzed in close loop form and compared based on a power settling product efficiency criteria. The nested gain boosted cascode OTA topology was found to have the best efficiency under high supply voltage, high overdrive voltage or low supply voltage, low overdrive voltage. Finally, a 2V 20Msample/s 11-bit pipelined ADC was designed as an example to demonstrate the benefits of the nested cascode OTA application to low voltage pipelined ADC design. The size of the ADC stages was optimally scaled to achieve low power consumption. The full ADC was simulated on the behavior model level by using Matlab Simulink. Cadence simulations and the Peregrine 0.5um SOS device models were used to verify critical components of the ADC further demonstrating feasibility.Electrical Engineering Technolog

Improving Differentiable Architecture Search via Self-Distillation

Differentiable Architecture Search (DARTS) is a simple yet efficient Neural Architecture Search (NAS) method. During the search stage, DARTS trains a supernet by jointly optimizing architecture parameters and network parameters. During the evaluation stage, DARTS discretizes the supernet to derive the optimal architecture based on architecture parameters. However, recent research has shown that during the training process, the supernet tends to converge towards sharp minima rather than flat minima. This is evidenced by the higher sharpness of the loss landscape of the supernet, which ultimately leads to a performance gap between the supernet and the optimal architecture. In this paper, we propose Self-Distillation Differentiable Neural Architecture Search (SD-DARTS) to alleviate the discretization gap. We utilize self-distillation to distill knowledge from previous steps of the supernet to guide its training in the current step, effectively reducing the sharpness of the supernet's loss and bridging the performance gap between the supernet and the optimal architecture. Furthermore, we introduce the concept of voting teachers, where multiple previous supernets are selected as teachers, and their output probabilities are aggregated through voting to obtain the final teacher prediction. Experimental results on real datasets demonstrate the advantages of our novel self-distillation-based NAS method compared to state-of-the-art alternatives.Comment: Accepted by Neural Network

Robust Neural Architecture Search

Neural Architectures Search (NAS) becomes more and more popular over these years. However, NAS-generated models tends to suffer greater vulnerability to various malicious attacks. Lots of robust NAS methods leverage adversarial training to enhance the robustness of NAS-generated models, however, they neglected the nature accuracy of NAS-generated models. In our paper, we propose a novel NAS method, Robust Neural Architecture Search (RNAS). To design a regularization term to balance accuracy and robustness, RNAS generates architectures with both high accuracy and good robustness. To reduce search cost, we further propose to use noise examples instead adversarial examples as input to search architectures. Extensive experiments show that RNAS achieves state-of-the-art (SOTA) performance on both image classification and adversarial attacks, which illustrates the proposed RNAS achieves a good tradeoff between robustness and accuracy

A Survey on Model Compression for Large Language Models

Large Language Models (LLMs) have revolutionized natural language processing tasks with remarkable success. However, their formidable size and computational demands present significant challenges for practical deployment, especially in resource-constrained environments. As these challenges become increasingly pertinent, the field of model compression has emerged as a pivotal research area to alleviate these limitations. This paper presents a comprehensive survey that navigates the landscape of model compression techniques tailored specifically for LLMs. Addressing the imperative need for efficient deployment, we delve into various methodologies, encompassing quantization, pruning, knowledge distillation, and more. Within each of these techniques, we highlight recent advancements and innovative approaches that contribute to the evolving landscape of LLM research. Furthermore, we explore benchmarking strategies and evaluation metrics that are essential for assessing the effectiveness of compressed LLMs. By providing insights into the latest developments and practical implications, this survey serves as an invaluable resource for both researchers and practitioners. As LLMs continue to evolve, this survey aims to facilitate enhanced efficiency and real-world applicability, establishing a foundation for future advancements in the field

Antipoisoning Nickel-Carbon Electrocatalyst for Practical Electrochemical CO2 Reduction to CO

The feasibility of utilizing electrochemical reduction of CO2 (CO2RR) to close the global carbon cycle is hindered by the absence of practical electrocatalysts that can be adopted in large CO2 emitting sources with impurities. To address this, we use density functional theory (DFT) calculations to design a strategy to develop Ni coordinated graphitic carbon shells (referred as Ni@NC-900) catalyst. This strategy not only prolongs stability and endows antipoisoning properties of the catalyst but also reforms the electronic structure of the outer graphitic carbon shell to make it active for CO2RR. As a result, Ni@ NC-900 demonstrates a high conversion of CO2 to CO with a Faradaic efficiency (FECO) of 96% and a partial current density for CO (jCO) of ∼−17 mA cm−2 at an applied potential of −1 V versus reversible hydrogen electrode (RHE). This activity can be further scaled up to attain a jCO of ∼30 mA cm−2 for 18 h at a cell voltage of 2.6 V in a high-throughput continuous gas diffusion electrode (GDE) system. In addition to exhibiting high activity and stability, Ni@NC-900 displays exceptional tolerance toward impurities (from SOx, NOx, CN−), highlighting the suitability of these rationally designed catalysts for large-scale application in fossil-fuel based power plantsThe work was supported by the Australian Research Council (ARC) under the Laurate Fellowship Scheme FL-140100081, Discovery Early Career Researcher Award DE170100375 and funding from the UNSW Digital Grid Futures Institute, UNSW Sydney under a crossdisciplinary fund scheme